AD-A104  741  TEXAS  A  ANO  N  UNIV  COLLEGE  STATION  INST  Of  STATISTICS  F/O  i?/, 

or  LINEAR  OCPCNOENcJ 2g  «OBAC  i^.u, 
UNCLASSIFIED  TR-N-27  DAA02N  80-C-0070 

_  NL 


ADA104741 


TEXAS  A&M  UNIVERSITY 

COLLECE  STATION,  TEXAS  77t&3  \ 


Institute  of  Statistics 
Phone  713  -  445-3141 


o 

Uu» 


I 


L 


COMMENTS  ON 
THE  ^MEASUREMENT  OFJ.INEAR  DEPENDENCE  AND 

FEEDBACK  ^ETWEEN  J1ULTIPLE  TIME  SERIES^ 

JV)  Ini'  0  c' ' 1 1 «-  K  r -  ^ 


1  '■ 


‘Emanuel  Parzen  • 
Institute  of  Statistics 
Texas  A&M  University 


/Vi  T  R-ti 


7  I 


ll 


Technical  j(epert^j»or  N-27^ 
July  1981 

,  f  c,  r,  :  /  _  fS.  "  /  (7 *1 


,l'i\  Ju1^  l 


*  z’*  ■ 

Vv 


C/Ll 


V' 


Texas  A&M  Research  Foundation 
Project  No.  4226T 


"Robust  Statistical  Data  Analysis  and  Modeling" 


Sponsored  by  the  Office  of  Naval  Research 


Professor  Emanuel  Parzen,  Principal  Investigator 


Approved  for  public  release;  distribution  unlimited. 


819 _ 3  0  (i^l 


& 


SECURITY  CLASSIFICATION 


PAGE  fWhan  Dmm  InlimO 


REPORT  DOCUMENTATION  PAGE 


RBAD  INSTRUCTIONS 
BEFORE  COMPLETtNO  FORM 


4.  TITLE  Can 4  Mtlllt) 

Comments  On  "The  Measurement  of  Linear 
Dependence  and  Feedback  Between  Multiple 
Time  Series" 


7.  AUTHORS) 

Emanuel  Parzen 


PERFORMING  OROANI2ATION  NAME  ANO  ADDRESS 

Texas  A&M  University 
Institute  of  Statistics 
College  Station.  TX  77843 


n.  CONTROLLING  OFFICE  NAME  AND  AOORESS 


S.  TYRE  OF  REPORT  *  PERIOD  COVEREO 

Technical 


»»■  NUM.cn  or  raoc* 

10  (ten 


MONITORINO  AGENCY  NAME  *  ADDRESS <1/  fftf/ereni  from  CimtmlHn*  Office)  tS.  SECURITY  CLASS.  (•*  IM#  re pmtj 

Unclassified 


IS.  OtSTRlRUTION  STATEMENT  (•!  th  it  Kfrt) 


Approved  for  public  release;  distribution  unlimited. 


17.  DISTRIBUTION  STATEMENT  f#l  th#  #h#tr#et  Mteretf  Iff  *(•«*  JO,  II  mttfwmt  Dm  XffpmtJ 


IS.  KEY  WOROS  (CwrtWff  #n  revere#  alR  II  wiMMity  ml  MmNS  Sr  WMft  iw#Mr) 

Multiple  time  series  analysis,  spectral  analysis,  autoregressive 
models,  information  numbers,  feedback. 


COMMENTS  ON 


"THE  MEASUREMENT  OF  LINEAR  DEPENDENCE  AND 
FEEDBACK  BETWEEN  MULTIPLE  TIME  SERIES" 

BY  JOHN  GEWEKE 

by  EMANUEL  PARZEN 

Institute  of  Statistics 
Texas  A&M  University 

I  would  like  to  congratulate  Professor  Geweke  on  an 
interesting  paper.  I  believe  its  most  valuable  contribution 
is  to  stimulate  us  to  develop  improved  methods  for  modeling 
multiple  time  series  with  the  aim  of  determining  which  variables 
are  significantly  related. 

The  problem  of  modeling  multiple  time  series  is  one  on 
whose  theory  I  have  written  extensively  in  Parzen  (1967) ,  (1967a) 
(1969),  (1977),  and  Parzen  and  Newton  (1900).  I  would  like  to 
show  how  results  and  notation  from  these  papers  help  us  to 
derive  and  clarify  the  results  presented  by  Geweke. 

Let  X(t)  and  Y(t)  be  multiple  time  series,  with  zero  means, 
jointly  normal,  and  jointly  covariance  stationary.  To  study 
the  relations  between  X(-)  and  Y(-).  one  models 


The  covariance  matrix  R(v)  -  E[Z'(t)  Z(t+v)]  is  assumed 
to  be  summable  so  that  the  spectral  density  matrix 

f (w)  -  I  e-2nivw  R(v) ,  0<w<l , 

yjm—co 

exists.  Then  R(v)  -  ^e2ltivw  f(w)  dw. 
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The  joint  covariance  and  spectral  density  matrices  of  X  and  Y 
are  described  by  the  blocks  in  the  partitioned  matrices 


R(v)  «  |  Hxx(v)  Rxy(v) 


f(w)  =  I  fxx(w)  fXY(w) 
fYX^  fYY^w^ 


Autoregressive  analysis  models  Z(t)  by  a  joint  infinite  order 
autoregressive  scheme: 


"x(t)~ 

X(t- 

-1) 

X(t-m) 

=  Ay  V  (1) 

*^» » »*^  y (nO 

Y(t) 

X,  I 

Y(t- 

•1) 

A,  I 

Y(t-m) 

ax,y(J) 


n(t) 


+  n(t), 

AjyjCj)  AxyO) 

AYX(j)  ayy<J> 

nx  (t)"]  joint  innovations 
(t) 


A  preferred  notation  for  E  is  E(X, Y|x“ ,Y") . 

We  call  n(t)  the  joint  innovations,  and  E  the  Joint 

innovation  covariance  matrix.  We  can  define  n(t)  «*  infinite 
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Etn' (t)n(t)]  | 

eyx  fyY 
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memory  prediction  errors 

n(t)  =  /x(t)\-Ef/k(t)| 


X(t) \"E  «(t)| 
Yd)/  1\Y(D/ 


X ( l: - 1 ) ,  Y(t-l) . . .  .X(t-in)  ,Y(t-m) , 


The  joint  innovations  should  be  contrasted  with  the 
individual  innovations 

X  (t)  =  X(t)  -  E [X( t ) | X(t-l) . X(t-m) , . . . J 

Y  (t)  =  Y(t)  E[Y(t) | Y(t-l) , . . . , Y(t-m) , . . .] 
which  provide  individual  infinite  order  autoregressive  models 


X(t)  =  Ax|x(1)  X(t-1)+. . .+AX|X  X(t-m)+. . .+X(t) 

Y(t)  =  AY|Y(1)  Y(t-1)+. . .+AY|Y  Y(t-m)+. . .  +  Y(t) . 

The  individual  innovation  covariance  matrices  are  denoted 

Ex  =  E(X|X~)  -  E[{X(t)}’  X(t)J , 

ZY  -  E(Y | Y~)  -  E[{Y(t)}’  Y(t) ] . 

The  innovation  innovations  are  defined  to  be  the  joint 
innovations  of  the  joint  time  series  rx(t)*j  of  individual 

LyooJ 

innovations.  A  remarkable  theorem  is  that  the  innovation 
innovations  are  identical  with  the  joint  innovations.  Thus  in 
practice,  one  can  determine  the  joint  innovations  of  X(t)  and 
Y(t)  by  first  "prewhitening"  them  to  form  X(t)  and  Y(t),  whose 
joint  innovations  are  then  determined. 


The  general  theory  of  multiple  time  series  discussed  below 
is  phrased  in  terms  of  general  stationary  time  series  X(t)  and 
Y(l).  However  it  works  best  in  practice  when  applied  to  time 
series  which  have  been  somewhat  pre-whitened. 

In  several  papers  on  model  identification,  Parzen  shows 
that  the  individual  innovations  of  a  time  series  are  essentially 
unique,  while  the  whitening  filter  which  generates  them  may  be 
expressed  in  diverse  ways  as  a  series  of  filters  representing 
detrending,  deseasonalizing,  and  innovations  operations. 

To  model  Y(t)  we  compare  the  properties  of  the  prediction 
errors,  and  prediction  error  covariance  matrices,  corresponding 
to  five  sets  of  explanatory  variables: 

Prediction  error  Covariance  Matrix 

(YV | Y_) (t)  =  Y(t)  -  E [ Y ( t) | Y(t-l) . Y(t-m) ]  I(Y|Y‘) 

(YV|X",Y")(t)  -  Y(t)  -  E [Y(t) | X(t-l) . X(t-m) ,  E(Y|X",Y") 

Y(t-l) , . . . , Y(t-m) ] 

(Yv|X+,Y')(t)  =  Y(t)  -  EtY(t) |x(t) ,X(t-l) . X(t-m) ,  E(Y|X+,Y') 

Y(t-l) . Y(t-m)  ] 

(YV|X)(t)  -  Y(t)  -  E[Y(t)|X(s),  -»<s«*>]  E(Y|X) 

(Yv|X,Y')(t)  -  Y(t)  -  E[Y(t)|X(s),  -»<s«,  E(Y|X,Y") 

Y(t-l)  ,  Y(t-2) . Y(t-m)  ] 

It  should  be  noted  that  after  a  conditioning  sign  | ,  X” 
represents  the  past  of  X,  X+  the  past  and  present  of  X,  and  X 
the  past,  present,  and  future  of  X. 


-  J- 


The  spectral  densities  of  the  various  error  series  are 
denoted  fyV|y-(w),  fyv  |  x“  ,  Y"  ^  '  fYv|X+,Y"^’  fYv|X^’ 


f  Yv  |  X,  Y~  (w) 


(1)  Y  |  Y~ 


E(Y|Y~)  is  the  individual  innovation  covariance 


matrix  Ey;  (Yv|Y”)(t)  is  white  noise. 

(2)  Y 1 X~ ,  Y~ :  E (Y I X~ .  Y')  is  the  block  Eyy  in  the  joint 

innovation  covariance  matrix;  (YV|X~,  Y  ) (t)  is  white  noise. 

(3)  Y|X+,  Y~ :  Parzen  (1967)  shows  (p.  401) 


(Yv|X+,Y‘)(t)  =  (Yv |X~  ,Y”)  (t)  -  Lyx  E^'*  (Xv|X',Y')(t); 

■1 
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rV,„- 


E(Y|X+,Y")  =  Eyy  -  E 


YX  EXX  *  EXY 


(4)  Y | X:  Parzen  (1967)  shows  that  a  joint  autoregressive 
model  for  X(t)  and  Y(t)  provides  f(w)  from  which  one  can 
compute  the  statistical  parameters  of  the  representation 

Y(t)  -  (Yy|X)(t)  +  (Yv | X) (t) , 
by 

(Yg|X)(t)  -  B(L)  X  (t),  B(z)  -  ?  B(k)zk 

k--“ 

B(e27,iw)  -  fyx(w)  fxx~1(w) 


fyv  |x^w^  m  fyyCw)  “  ^ 


XX  1(w)  fXY(w) 


E(Y|X)  -  ^lfYv|X^  dw< 


In  practice,  to  estimate  the  time  domain  coefficients  B(k) 

a  O  iw  4 

from  an  estimator  B  (e  )  one  uses  regression  methods. 
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(5)  Y | X ,  Y~ :  As  Geweke  shows,  find  an  autoregressive  model 
for  (Y^|X)(t); 

gyv(x(L)  (Y v  | X)  ( t)  =  c(t). 

It  generates  a  white  noise  sequence  r(t)  which  can  be  identified 
with  (Yv |X,Y~) (t) .  Further  a  model  for  Y(t)  as  a  function  of 
X(s)  ,  and  Y(s),  s<t-l  is  given  by 

gYV|X(L)  V(t)  =  gYv(x(L)B(L)  X(t)  +  e(t) ; 

E(Y|X,  Y  )  =  *  covariance  matrix  of  e(t). 

The  time  domain  coefficients  of  the  filter  with  input  X(t) 
have  Fourier  transform 

„  ,  -2TriwN  n,  -2iTiWv 

gYv|X  '  B(e  ' ‘ 

If  one  needs  only  the  log  determinant  of  it  can  be  calculated 
without  fitting  an  autoregressive  scheme: 

log|£c  |  ■  J1  log|fy\>|X(w)|  dw. 

One  need  not  actually  calculate  the  spectral  density  since 
Geweke 's  Theorem  1  shows  that 

In  |  £ (Y | X, Y_) |  -  £n| E(X,Y|X“,Y") |  -  In  |E(X|X‘)| 

The  meaning  of  the  various  definitions  of  feedback,  and  the 
formulas  for  them  given  by  Geweke' s  Theorem  1,  is  easily 
understood  if  one  employs  the  notation  we  have  introduced. 


V 


Measure  of  linear  dependence  (or  information)  y  „ 

=  In  det  >:  (X  |  X")  -  In  det  E(X|X~,Y) 

=  In  det  >:( Y|Y")  -  In  det  E(Y|Y",X) 

Measure  of  instantaneous  linear  feedback :  • Y  = 

=  in  det  >; (X | X“ ,  Y")  -  in  det  E(X|X',Y+) 

=  in  det  J!(Y|X*,Y")  -in  de-tZ(Y|X+,Y") 

Measure  of  linear  feedback  from  Y  to  X:  * 

=  in  det  E(X|X_)  -  in  det  £(X|X“,Y") 

=  in  det  Z(Y|X+,Y")  -  in  det  E(Y|X,Y“) 

Measure  of  linear  feedback  from  X  to  Y:  F^y  * 

=  in  det  E (Y | Y“)  -  in  det  E(Y|Y',X") 

=  In  det  E (X | X- , Y+)  -  in  det  E(X|X‘,Y) 

Theorem  1  in  Geweke's  paper  shows  that  there  is  a  crucial 

identity  from  which  the  equivalence  of  the  foregoing  definitions 

follows  immediately: 

in  det  E(X|X-,Y+)  +  in  det  E(Y|X",Y") 

-  in  det  E(X|X~ ,Y)  +  in  det  E(Y | Y-) 

=  in  det  E  (X,  Y  |  X-  ,  Y~) 

These  feedback  measures  seem  to  me  to  be  most  clearly 
interpreted  as  measuring  the  significance  of  various  variables 
as  independent  variables  in  a  model  for  Y(t). 
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Variables 

In 

Variables  tested 

Measure 

in  Model 

Words 

for  inclusion 

Add 

FX  >Y 

Y" 

Past  Y 

Y'.X" 

Past  X 

# 

F  y  y 

Y“,X" 

Past  Y, 

Y"  X+ 

Present  X 

X  *  i 

Past  X, 

» 

fy>x 

Y',X+ 

Past  Y, 
Past  X, 

Y'.X 

Future  X 

Present 

X 

fx,y 

Y' 

Past  Y 

Y",X 

All  X 

This  table  also  exhibits  the  information  decomposition  of  the 
identity: 

FX,Y  FX+Y  +  FX-Y  +  FY-X 

One  uses  y  to  compare  the  hypotheses  Hq:  model  Y  by  Y  ; 

:  model  Y  by  Y~,X.  In  addition  one  should  compute  F(Y|X) 
in  order  to  compare  the  hypotheses  Hq:  model  Y  by  X;  H^:  model 
Y  by  X,Y‘. 

In  any  empirical  multiple  time  series  analysis,  one  should 
compute,  and  report,  F. Fy,  F  (the  individual  and  joint  innovation 
covariances) .  Then  one  should  compute  (and  test  for  significant 
difference  from  zero) 

FX-*-Y  “  *-n  de't  EYY 

F^^y  3  “  fw  dct  (I  “  Fyy  Fyj£  F  y  y  F^y) 

=  In  dzt  Fjj  +  In  de.t  F^  -  In  de.t  F 

Fy>x  ”  In  de.t  F^  -  In  dit  F^ 

The  computation  of  these  determinants,  and  additional 
insight  into  the  relations  between  variables,  could  be  attained 
by  computing  the  eigenvalues  and  eigenvectors  of  the  matrices, 


such  as  y-1  (Y  |  Y' ,  X")  >:(Y  |Y"  ,X+)  ,  whose  log  determinants  are 
being  calculated.  The  eigenvalues  can  bo  Interpreted  in  terms 
of  various  canonical  correlations  (see  Parzen  and  Newton  (19/9)) 

Finally,  to  estimate  (from  data)parameters  such  as  T, , 

>^,  and  y y .  it  is  strongly  recommended  that  one  use  approximatin 
autoregressive  schemes,  and  order-determining  criteria  such  as 
CAT  (see  Parzen  (1974),  (1977)). 

As  with  any  excellent  piece  of  research,  Geweke’s  paper 
raises  many  open  questions,  some  of  which  have  been  alluded  to 
in  my  discussion. 

The  definition  of  the  feedback  spectral  measure  fy-t-X^w^ 
given  by  Geweke  is  impressive.  Only  experience  can  show  us 
whether  it  should  be  routinely  computed  in  empirical  research; 
it  may  suffice  to  use  as  the  feedback  spectral  measure 

f^  y(^^  fw  X  —  f y y  ( W )  ^XX  ( ^ )  ^xY 

computed  using  canonical  spectral  analysis  (see  Brillinger 
(1981) ,  chapter  10  for  references  to  this  literature) . 
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